Gesture shaking recognition method and apparatus, and gesture recognition method

ABSTRACT

A gesture shaking recognition method includes acquiring two adjacent frames of gesture images of a gesture (S 100 ), selecting a tracking point in each of the two adjacent frames of gesture images (S 200 ), determining positional information of a maximum value pixel point in each of the two adjacent frames of gesture images based on the tracking point (S 300 ), and determining whether gesture shaking occurs (S 400 ) based on the positional information of the maximum value pixel point in each of the two adjacent frames of gesture images and positional information of the tracking point in each of the two adjacent frames of gesture images. The tracking point in each of the two adjacent frames of gesture images may correspond to a same position on the gesture.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims benefit of the filing date of Chinese PatentApplication No. 201810264119.8 filed on Mar. 28, 2018, the disclosure ofwhich is hereby incorporated in its entirety by reference.

TECHNICAL FIELD

This disclosure relates to image recognition technology, and inparticular, to a gesture shaking recognition method and apparatus, and agesture recognition method.

BACKGROUND

In usage scenarios of virtual reality (VR) and other applicationdeveloped with gestures, there is possibility of hand shaking duringhand movement. In the prior art, the hand shaking may be suppressedrelying on a single standard or by in a full range calculation. However,such a method of using a single standard or a full range calculation cannot only make the standard in the calculation process not stable, butalso result in incorrect calculation results and increases calculationtime. When suppression of hand shaking is based on the full rangecalculation, since the entire gesture area needs to be considered andsearched, the calculation is greatly increased, thereby resulting inpoor human-computer interaction experience.

BRIEF SUMMARY

An embodiment of the present disclosure provides a gesture shakingrecognition method. The gesture shaking recognition method may includeacquiring two adjacent frames of gesture images of a gesture, selectinga tracking point in each of the two adjacent frames of gesture images,the tracking point in each of the two adjacent frames of gesture imagescorresponding to a same position on the gesture, determining positionalinformation of a maximum value pixel point in each of the two adjacentframes of gesture images based on the tracking point, and determiningwhether gesture shaking occurs based on the positional information ofthe maximum value pixel point in each of the two adjacent frames ofgesture images and positional information of the tracking point in eachof the two adjacent frames of gesture images.

Optionally, determining whether gesture shaking occurs based on thepositional information of the maximum value pixel point in each of thetwo adjacent frames of gesture images and the positional information ofthe tracking point in each of the two adjacent frames of gesture imagescomprises determining a first difference between the positionalinformation of the tracking point in each of the two adjacent frames ofgesture images, determining a second difference between the positionalinformation of the maximum value pixel point in each of the two adjacentframes of gesture images, and determining whether the gesture shakingoccurs based on the first difference and the second difference.

Optionally, determining whether the gesture shaking occurs based on thefirst difference and the second difference comprises determining thatthe gesture shaking occurs when a product of the first difference andthe second difference is less than zero.

Optionally, determining whether the gesture shaking occurs based on thefirst difference and the second difference comprises determining thatthe gesture shaking occurs when a product of the first difference andthe second difference is greater than or equal to zero, an absolutevalue of the first difference is less than a first threshold, and anabsolute value of the second difference is less than a second threshold.In one embodiment, the first threshold is 7 and the second threshold is5.

Optionally, determining the positional information of a maximum valuepixel point in each of the two adjacent frames of gesture images basedon the tracking point comprises sampling each of the two adjacent framesof gesture images to obtain at least one detection image for each of thetwo adjacent frames of gesture images, the two adjacent frames ofgesture images being as original detection images, constructing adetection area centering on the tracking point in each of the at leastone detection image and each of the original detection images, andcomparing all pixel values of pixel points in the detection areasequentially to determine the maximum value pixel point in each of thetwo adjacent frames of gesture images.

Optionally, comparing all pixel values of pixel points in the detectionarea sequentially to determine the maximum value pixel point in each ofthe two adjacent frames of gesture images comprises traversing a currentpixel point in a detection area in one of the at least one detectionimage and the original detection images, acquiring a plurality of pixelpoints adjacent to the current pixel point traversed, the current pixelpoint being a center of the plurality of pixel points, acquiring anotherpixel point and another plurality of pixel points adjacent to theanother pixel point in each of the remaining detection image andoriginal detection images, the another pixel point corresponding to thecurrent pixel point, and the another plurality of pixel pointscorresponding to the plurality of pixel points adjacent to the currentpixel point respectively, and determining whether a pixel value of thecurrent pixel point is the largest among pixel values of the pluralityof pixel points adjacent to the current pixel point, a pixel value ofthe another pixel point and pixel values of the another plurality ofpixel points in each of the remaining detection image and originaldetection images.

Optionally, the gesture shaking recognition method further includes,when the pixel value of the current pixel point is the largest,extracting positional information of the current pixel point into anextreme value array, the current pixel point being an extreme valuepixel point, and the position information of the current pixel pointbeing an element of the extreme value array.

Optionally, the gesture shaking recognition method further includessequentially comparing pixel values corresponding to elements in theextreme value array to determine the positional information of themaximum value pixel point in each of the two adjacent frames of gestureimages.

Optionally, when the extreme value array has zero element, the positioninformation of extreme value pixel point is re-determined based on thetracking point.

Optionally, sequentially comparing pixel values corresponding toelements in the extreme value array to determine the positionalinformation of the maximum value pixel point in each of the two adjacentframes of gesture images comprises traversing all elements of theextreme value array, the number of elements in the extreme value arraybeing greater than one, determining a distance between each pixel pointcorresponding to each of the elements and the tracking point, andcomparing the distance between each pixel point corresponding to each ofthe elements and the tracking point to acquire a pixel point having thesmallest distance, and the pixel point having the smallest distancebeing the maximum value pixel point.

Optionally, the gesture shaking recognition, before acquiring twoadjacent frames of gesture images of the gesture, includes acquiring twoadjacent frames of original images of the gesture and constructing asquare area as the detection area with a center being the tracking pointand a side length being one-fifth of a side length of an original imagebox.

Optionally, another example of the present disclosure includes a gesturerecognition method comprising the gesture shaking recognition methodaccording to one embodiment of the present disclosure to determinewhether a gesture shaking occurs.

Optionally, the gesture recognition method further includes, when thegesture shaking occurs, for the two adjacent frames of images comprisinga previous frame of image and a next frame of image, the next frame ofimage is processed by image parameters corresponding to the previousframe of image.

Another example of the present disclosure is a gesture shakingrecognition apparatus. The gesture shaking recognition apparatusincludes an image acquisition module configured to acquire two adjacentframes of gesture images of a gesture, a tracking point selection moduleconfigured to select a tracking point in each of the two adjacent framesof gesture images, wherein the tracking point in each of the twoadjacent frames of gesture images corresponding to a same position onthe gesture, a maximum value pixel point acquisition module configuredto determine, based on the tracking point, positional information of amaximum value pixel point in each of the two adjacent frames of gestureimages, and a determination module configured to determine whether thegesture shaking occurs based on the positional information of themaximum value pixel point and the position information of the trackingpoint in each of the two adjacent frames of the gesture images.

Another example of the present disclosure is a computer readable storagemedium storing a computer program that, when executed, implements thesteps of the gesture shaking recognition method according to oneembodiment of the present disclosure.

Another example of the present disclosure is a gesture shakingrecognition apparatus comprising a processor and a memory, the memorybeing configured to store a computer program that, when the computerprogram is executed by the processor, implements the steps of thegesture shaking recognition method according to one embodiment of thepresent disclosure.

Another example of the present disclosure is a gesture recognitionapparatus, comprising a camera device and the computer readable storagemedium according to one embodiment of the present disclosure or thegesture shaking recognition apparatus according to one embodiment of thepresent disclosure being connected to the camera device, wherein thecamera device is configured to collect an image stream comprising thetwo adjacent frames of the gesture images

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of thedisclosure are apparent from the following detailed description taken inconjunction with the accompanying drawings in which:

FIG. 1 is a flow chart of an embodiment of a gesture shaking recognitionmethod according to one embodiment of the present disclosure;

FIG. 2(a) is an original image of a gesture image according to oneembodiment of the present disclosure;

FIG. 2(b) is a YCrCb image with Cr channel extracted according to oneembodiment of the present disclosure;

FIG. 2(c) is an image of a hand contour segmented using an otsualgorithm according to one embodiment of the present disclosure;

FIG. 2 (d) is a contour image of the hand calculated from FIG. 2 (c);

FIG. 3 is a flowchart of a gesture shaking recognition method accordingto one embodiment of the present disclosure;

FIG. 4 is a flowchart of a gesture shaking recognition method accordingto one embodiment of the present disclosure;

FIG. 5(a) is a detection area constructed in each of three detectionimages according to one embodiment of the present disclosure;

FIG. 5(b) is an enlarged image of a detection area in each of threedetection images according to one embodiment of the present disclosure;

FIG. 5(c) is a nine-square grid pixel image constructed by correspondingadjacent pixel points in each of three detection images according to oneembodiment of the present disclosure;

FIG. 6 is a flowchart of a gesture shaking recognition method accordingto one embodiment of the present disclosure;

FIG. 7 is a schematic flowchart of gesture recognition method accordingto one embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a gesture shakingrecognition apparatus according to one embodiment of the presentdisclosure.

DETAILED DESCRIPTION

The present disclosure will be described in further detail withreference to the accompanying drawings and embodiments in order toprovide a better understanding by those skilled in the art of thetechnical solutions of the present disclosure. Throughout thedescription of the disclosure, reference is made to FIGS. 1-8. Whenreferring to the figures, like structures and elements shown throughoutare indicated with like reference numerals.

Those skilled in the art can understand that, the singular forms “a/an”,“the” and “said” may include the plural forms unless specificallystated. It is to be further understood that the phrase “comprise” or“include” used in the disclosure indicates the inclusion of thefeatures, integers, steps, operations, components, and/or groupsthereof, but does not exclude the possibility of including or adding oneor more other features, integers, steps, operations, components, and/orgroups thereof. It should be understood that when an element is referredto as being “connected” or “coupled” to another element, it can bedirectly connected or coupled to the other element, or the intermediateelements exist. Further, “connected” or “coupled” as used herein mayinclude either a wireless connection or a wireless coupling. The term“and/or” used herein includes all or any of the elements and allcombinations of one or more of the associated list.

Those skilled in the art will appreciate that all terms (includingtechnical and scientific terms) used herein have the same meaning ascommonly understood by one of ordinary skill in the art to which thedisclosure belongs, unless otherwise defined. It should also beunderstood that terms such as those defined in a general dictionaryshould be understood to have the meaning consistent with the meaning inthe context of the prior art, and will not be explained as idealized orexcessive formal meaning, unless specifically defined as here.

A gesture shaking recognition method is provided according to oneembodiment of the present disclosure. As shown in FIG. 1, the gestureshaking recognition method includes steps S100 to S400.

Step S100 includes acquiring two adjacent frames of gesture images of agesture.

Step S200 includes selecting a tracking point in each of the twoadjacent frames of gesture images, where the selected tracking points inthe two adjacent frames of gesture images correspond to a same positionon the gesture.

Step S300 includes determining positional information of a maximum valuepixel point, that is, a pixel point corresponding to a maximum pixelvalue, in each of the two adjacent frames of gesture images based on thetracking points.

Step S400 includes determining whether the gesture is shaking based onthe corresponding positional information of the maximum value pixelpoints and the positional information of the tracking points in the twoadjacent frames of gesture images.

In the implementation process according to one embodiment of the presentdisclosure, a camera device captures images, and the gesture shakingrecognition method is performed based on the images captured by thecamera device. In the implementation process, the system for performingthe gesture shaking recognition method is initialized, and the initialparameters of the system are adjusted to facilitate subsequentprocessing of the images based on the system with the adjustedparameters. After the system initialization is completed, the parameterssuch as the frame rate of the camera device, the size of the videoimage, etc. are set, and some parameters of the system or the fixeddevice can be uniformly set. As such, the steps of image processing canbe reduced, thereby improving accuracy of the image processing.

Since the present disclosure is mainly related to the human-computerinteraction process in the AR virtual scene, the camera device mainlycaptures a gesture image stream of the user in the human-computerinteraction process, and acquires the gesture images from the gestureimage stream. In one embodiment, as shown in FIG. 2 (a), the imagestream is transmitted to an image processing determination device (agesture shaking recognition device), and two adjacent frames of gestureimages are acquired from the image stream (video stream) captured by thecamera device which includes the user's gesture image stream. In orderto facilitate determination of gesture shaking, and reduce cumbersomesteps and promote accuracy of gesture shaking determination, the gestureimages are pre-processed by some techniques such as Gaussian filteringor image noise reduction. In addition, the gesture images are processedas much as possible into images having a single color signal component.After the foregoing processes, the current frame of the gesture image isconverted into an YCrCb color space with the Cr channel extracted, asshown in FIG. 2(b)). Then, the image is binarized. On the basis ofbinarization, the area of the hand is segmented out of the gestureimage. For example, an otsu algorithm is used to segment the area of thehand, as shown in FIG. 2(c), and a contour of the hand is extracted fromthe area of the hand, as shown in FIG. 2(d)). The extracted contour ofthe hand is recorded as a hand contour. The area outside the handcontour and enclosed in the rectangular box is termed as an area of thegesture image box, that is, an area surrounding the hand contour imageand inside the outer box, wherein the gesture image is the image shownin FIG. 2(d), that is, the hand contour image in an embodiment of thepresent disclosure.

In order to facilitate comparison, it is necessary to extract areference point in the hand contour image. For example, the target is ahand, and the reference point is a tracking point in the embodiment ofthe present disclosure, so that each image has the same tracking pointas the reference point during processing, which facilitates the imagecomparison, determination, and recognition. That is, in the gestureshaking recognition method in the embodiment of the present disclosure,the tracking point of each frame of image corresponds to the sameposition on the target object (hand). Specifically, in the gestureshaking recognition method according to one embodiment of the presentdisclosure, since the hand can be regarded as motion of a semi-rigidobject, the shape of the hand in the image often changes due to theflexibility of the finger. As a result, the motion pattern is irregular.However, the wrist is less deformed in the image. During the movement ofthe hand, the wrist is relatively more stable comparing to the fingersdue to the characteristics of clothing shielding and small deformation.Therefore, the lower left corner of the hand contour can be selected asthe tracking point (handPos), and the tracking point records theposition of the hand gesture detected each time from the image.

After the tracking point is determined, the position of the hand gestureis determined based on the tracking point. A detection area isconstructed based on the tracking point as a reference, and a maximumpixel value in a frame of image is determined by traversing andcalculating the pixel values of the pixel points in the detection area.The maximum pixel value is converted into a gray value after thebinarization process described above, and the maximum pixel value is themaximum gray value in the detection region in the current frame ofimage. The positional information of the pixel corresponds to themaximum pixel value is the positional information of maximum value pixelpoint, and the maximum pixel value is the maximum pixel value in thecurrent frame of the hand gesture, and the positional information of thepixel point corresponding to the maximum pixel value is termed maxValue.The positional information of the pixels corresponding to the maximumpixel values in the two adjacent frames of gesture images respectively(i.e., positional information of the maximum value pixel point) iscompared to determine whether the gesture shaking occurs. In thedetermination process, whether the gesture shaking occurs is determinedbased on the difference between the position information of the maximumvalue pixel points of the two adjacent frames of gesture images and thedifference between the positions of the tracking points in the twoadjacent frames of gesture images. The detailed process thereof isdescribed later.

In the embodiments of the present disclosure, the adjacent two frames ofimages are interval frames for which the user presets for the gestureimage determination, and the adjacent two frames of images are twoadjacent frames in the gesture determination process. Therefore, theadjacent two frames of images may be two adjacent frames in the imagestream or two interval frames that are not adjacent to each other. Byprocessing and determination of some local pixels, it is possible toquickly and reliably determine whether the gesture shaking occurs.

Further, in one embodiment, as shown in FIG. 3, the step of determiningwhether or not the target is shaking based on the positional informationof the maximum value pixel points and the positional information of thetracking points in the two adjacent frames of images includes steps S410to S430 as below:

Step S410 includes comparing positional information of the trackingpoints in the two adjacent frames of images to determine a positionaldifference, a first difference, between the two tracking points;

Step S420 includes comparing positional information of the maximum valuepixel points in the two adjacent frames of images to determine apositional difference, a second difference, between the two maximumvalue pixel points;

Step S430 includes determining whether the gesture shaking occurs basedon the positional difference of the tracking points and the positionaldifference of the maximum value pixel points.

Furthermore, in one embodiment, the step of determining whether thegesture shaking occurs based on the positional difference of thetracking points and the positional difference of the maximum value pixelpoints includes the following:

When the product of the positional difference of the tracking points,that is, the first difference, and the positional difference of themaximum value pixel points, that is, the second difference, is less thanzero, it is determined that the target gesture shaking occurs.

Furthermore, in one embodiment, the step of determining whether thegesture shaking occurs based on the positional difference of thetracking points and the positional difference of the maximum value pixelpoints includes the following:

When the product of the positional difference of the tracking points andthe positional difference of the maximum value pixel points is greaterthan or equal to zero, and the absolute value of the positionaldifference of the tracking points is less than a preset positionalthreshold, which is a first threshold, and the absolute value of thepositional difference of the maximum value pixel points is less than apreset maximum threshold, which is a second threshold, it is determinedthat the target gesture shaking occurs.

On the basis of the foregoing, the position information of the maximumvalue pixel point in a frame of gesture image is determined on the basisof the tracking point. Specifically, in the process of determiningwhether the gesture shaking occurs, the image of the next frame isreceived by the imaging device and recorded as the next frame of theimage. The previous process is repeated to get new parameters such astracking point position and maximum value pixel point, and positionalinformation of the tracking point and the maximum value pixel point inthe next frame of image is recorded as handPosNext and maxValueNext. Thedistance between handPos and handPosNext is recorded as disHandPos,which is the first difference. The distance between maxValue andmaxValueNext is recorded as disMaxValue, which is the second difference.In the process of making a specific determination, the following processis implemented:

Specifically, the calculation method is

disHandPos=handPosNext-handPos,

disMaxValue=maxValueNext-maxValue.

Whether or not the hand shaking occurs is determined based on the valuesof disHandPos and disMaxValue.

For example, condition one: if disHandPos*disMaxValue>=0, it means thatmoving direction of the two points is the same, then furtherdetermination is needed. The further condition for determination iscondition two. Condition two: |disHandPos|<7 and |disMaxValue|<5. Whenthe condition 1 and the condition 2 are satisfied, the gesture isdetermined to be a shaking phenomenon. In addition, when the product ofthe values of the disHandPos and the disMaxValue does not satisfy thecondition 1, the gesture can be directly determined to be a shakingphenomenon. In the foregoing cases, when the condition one is satisfiedand the condition two is not satisfied, it can be determined that thegesture is not a shaking phenomenon, but a gesture movement. In theembodiments of the present disclosure, it should be noted that theinverse expression of the condition one is disHandPos*disMaxValue<0,that is, when disHandPos*disMaxValue<0, the condition 1 is notsatisfied. In addition, |disHandPos|<7 and |disMaxValue|<5. In oneembodiments provided by the present disclosure, the preset positionalthreshold is 7, and the preset maximum threshold is 5. It should benoted that the preset positional threshold and the preset maximumthreshold may also be set according to actual needs, that is, the presetpositional threshold being 7 and the preset maximum threshold being 5 donot limit the scope of the present disclosure.

Furthermore, in one embodiment, as shown in FIG. 4, the step ofdetermining the positional information of the pixel point correspondingto the maximum pixel value in the gesture image based on the trackingpoint includes the following steps S310 to S330:

Step S310 includes sampling the image to obtain at least one detectionimage of the gesture image, and using the gesture image as an originaldetection image.

Step S320 includes constructing a detection area, which uses thetracking point as a center, in each detection image.

Step S330 includes sequentially comparing all pixel values of the pixelpoints in the detection areas to determine an extreme pixel value of allthe detection areas.

Furthermore, in one embodiment, before the step of acquiring the twoadjacent frames of gesture images, the method includes obtaining twoadjacent frames of original images including the gesture andconstructing a rectangular region, which uses the tracking point as acenter and has a side length of one-fifth of the length of the originalimage box, as the detection region in each of the original images.

In one embodiment provided by the present disclosure, the gesture imageis used as the original detection image, and the at least one imageobtained by sampling the image is a detection image, so that a detectionarea can be constructed in both the original detection image and thedetection images. As such, any one of the original detection image andthe detection images can be traversed hereafter, and the correspondingpixel points in the detection areas of the remaining detection imagescan be acquired. The original image is an image obtained by binarizingthe area of the hand. As can be seen from FIG. 2(c), the original imageis an image that segments the area of the hand by an otsu algorithm.

In order to obtain accurate comparison, based on the image with theextracted Cr channel, the Cr image is down-sampled according todifferent multiples, and the detection images of Cr images of differentmultiples are obtained. At the same time, the detection range of imagepixels is reduced, and the preferred detection area is determined. Assuch, the accuracy of the pixels at different multiples is improved, andaccordingly, the accuracy of gesture recognition is improved. The Crimage of the original multiple is also the detection image. Therefore,in each of the detection images, a preset area centered on a trackingpoint is constructed as a detection area, and pixel values of pixels inthe detection area are sequentially compared, and an extreme pixel valuein the detection areas is determined. Further, the maximum pixel valueis determined based on the pixel of the extreme pixel value, and thespecific process is as described above and the following.

In one embodiment, in order to facilitate the processing, on the basisof filtering and extracting the Cr channel of the image, the image withextracted Cr channel is termed as image 0. The image 0 is thendown-sampled by 2 times and 4 times respectively to construct two scalesspaces and obtain two new images, that is, image 1 and image 2 (shown inFIG. 5 (a)). The preset sampling rule is 2 times and 4 timesdown-sampling. According to the foregoing description, the originalimage is an image with the hand area segmented by an otsu algorithm (asshown in FIG. 2(c)), and a detection rectangle using the tracking point(handPos) as the center and one-fifth of the width of the gesture framearea as the side length is constructed as the detection area. As shownin the black rectangular box in FIG. 5(a), three detection rectanglesare constructed in the three images respectively. The detection area isenlarged, as shown in FIG. 5(b), and the black dots in the figures arethe tracking points. The extreme value is found in the detection areaand is recorded as the extreme pixel value, and the positionalinformation of the pixel corresponding to the extreme pixel value isrecorded as maxValue. In the process of constructing the detection areasto find the extreme pixel value, the size of the detection areasdirectly affects the search of the extreme pixel value. The activity ofthe hand will cause the size of the detection area to change. In oneembodiment, the box with a side length of one-fifth of the width of thegesture box is the most suitable size. Adopting one-fifth of the widthof the gesture box can balance the detection speed and accuracy duringthe detection process. That is, the detection speed is fast enough, andat the same time, it is ensured that the detected gesture can accuratelydetermine whether the user's unintentional shaking or the user'sintentional movement occurs. The detection area is the area in thedetection image for detecting the extreme pixel value.

Further, in one embodiment, as shown in FIG. 6, step S330 ofsequentially comparing the pixel values of pixels in the detection areaand determining the extreme pixel value in the detection area includessteps S331 to S335.

Steps S331 includes traversing the pixel points in the detection area ofany one of the detection images;

S332 includes using a current pixel point traversed as a center toobtain a plurality of pixel points adjacent to the current pixel point.

S333 includes acquiring, in each of the remaining detection images, apixel point corresponding to the current pixel point and a plurality ofadjacent pixel points corresponding to the pixel points adjacent to thecurrent pixel point.

S334 includes comparing the pixel value of the current pixel point withthose of the pixel points adjacent to the current pixel point, the pixelpoint corresponding to the current pixel point and the adjacent pixelpoints corresponding to the pixel points adjacent to the current pixelpoint in the remaining detection images to determine whether the pixelvalue of the current pixel point is the largest.

S335 includes, when the pixel value of the current pixel point is thelargest, extracting the positional information of the current pixelpoint into a preset extreme value array and obtain an extreme valuearray. The current pixel point is determined as the extreme value pixelpoint, and the positional information of the current pixel point is anelement of the extreme value array.

Further, in one embodiment, after the step of determining the positionalinformation of the pixel point corresponding to the extreme pixel valuein the image based on the tracking point, the method includessequentially comparing the pixel values corresponding to the elements inthe extreme value array to determine positional information of the pixelcorresponding to the maximum pixel value in the image.

According to the foregoing description, an area enclosed by thedetection area in any one of the detection images may be traversed, andthe detection image may be the original detection image or a detectionimage generated by sampling the original image. When the detection imagetraversed is the original detection image, the pixel point correspondingto the current pixel point traversed and the pixel points correspondingto the pixel points adjacent to the current pixel are acquired in thedetection images generated by sampling the original image. When thetraversed image is any one of the detected images generated by samplingthe original image, a pixel point corresponding to the current pixelpoint traversed and pixel points corresponding to the pixel pointsadjacent to the current pixel point in the remaining detected images andthe original detection image are obtained. In order to make the pixelvalue obtained after the comparison more accurate, in the preferreddetection area, the pixel values of the pixel points in the samedetection area are sequentially traversed and compared, so that theobtained maximum pixel value is a maximum pixel value that is preferredand reflects shaking of the gesture. Therefore, in addition to this, itis preferable to traverse the image 1 from the second row and the secondcolumn to the 19th row and the 19th column. Preferably, in theembodiment of the present disclosure, the partial image size is 20*20pixels, and the traversing template is 3*3 pixels. Thus, it is necessaryto leave a circle of pixels at the edges of the image to avoid emptypixels during comparison. Accordingly, the image 1 is traversed from thesecond row and the second column to the 19th row and 19th column.

In addition, in one embodiment of the present disclosure, when thecriteria of the respective determination conditions are all set based onthe image 1, the image 1 is preferentially traversed to lower thedetection error. That is, in the implementation of the presentdisclosure, during the subsequent process of determining whether theshaking of the image occurs, if image 2 or image 0 is used as thetraversing image, since they are ½ or 4 times of image 1 respectively,the pixel value of image 2 is twice as large as that of image 1, and thepixel value of the image 0 is twice as small as that of the image 1. Thecondition 1 and the condition 2 for determining whether or not the imageshaking occurs are designed based on the pixel values of the image 1.When the magnitude of enlargement and reduction of the image is large,reconstruction or interpolation occurs between adjacent pixel values,thereby resulting in a large change in pixel values and loweringaccuracy of the pixel values. Therefore, when the criteria of thedetermination condition is fixed, if the pixels corresponding to thepixel values of other images are used for determination, there may be acase where the pixel values change too large and accuracy of the pixelvalues is low, thereby lowering credibility of the final determinationthat gesture shaking or moving occurs. When this is applied toconditions 1 and 2 or other steps related to conditions 1 and 2, theaccuracy of the final determination is not high and the error isrelatively large. Therefore, the image based on which the criteria ofthe determination are designed should be used as the image traversed.Since image 0 and image 2 are images at different magnification of image1, for pixel point 1 in each traversed image 1, there are correspondingpixel point 0 in image 0 and corresponding pixel point 2 in image 2.That is, the pixel point 0 is in image 0. The pixel point 1 is in image1 and corresponds to pixel point 0. The pixel point 2 is in image 2 andalso corresponds to pixel point 0. The pixel point 0, the pixel point 1,and the pixel point 2 respectively correspond to the names oridentifiers of the pixel points in the image 0, the image 1, and theimage 2, and may be replaced by other symbols.

In one embodiment, after the pixel point 0, pixel point 1, and pixelpoint 2 are found in image 0, image 1, and image 2 respectively, thenine-square grid images centered on pixel point 0, pixel point 1, andpixel point 2 respectively have a total of 27 pixels (As shown in FIG.5(c)). It is then determined whether the value of pixel 1 is the largestamong the 27 pixel values. If so, pixel 1 is added to the preset extremevalue array, and the extreme value of pixel 1 is an element of thepreset extreme value array. Accordingly, the elements of the presetextreme value array are obtained. It should be noted that, in theembodiment of the present disclosure, if the image 0 is used as thetraversed image, and the determination condition is also designed basedon the image 0, when the image 0 is traversed, there are pixel points inthe image 1 and the image 2 corresponding to the current pixel in theimage 0 traversed. In combination with the foregoing description, theimage based on which the determination condition is designed is used asthe image traversed, thereby improving the accuracy of the gesturedetermination. As such, the impact of reconstruction and interpolationcalculation in other images on the result of image determination can beavoided. In addition, in the embodiment of the present disclosure, theimage 1 is used as the traversed image. The condition of the imagedetermination is designed based on the image 1. Furthermore, at the sametime, the down-sampling of the image 1 is equivalent to denoising image0 once again, thereby preventing the noise in the image from influencingthe determination result. Therefore, in one embodiment of the presentdisclosure, image 1 is used as the traversed image.

After the traversing of the detection area is completed, the elementsfor the preset extreme value array are obtained and extracted into thepreset extreme value array. Accordingly, an extreme value array isobtained. The elements in the extreme value array are sequentiallycompared to obtain a pixel point corresponding to the maximum pixelvalue in the image. The specific process is described later.

Further, in one embodiment, after the process of determining a maximumpixel value of the image based on the tracking point, the methodincludes:

When the number of elements in the preset extreme value array is zero,the positional information of the pixel corresponding to the maximumpixel value in the image is determined again based on the trackingpoint.

When the number of elements in the preset extreme value array is zero,combined with the foregoing description, it is indicated that when theimage 1 is traversed, the pixel having the largest pixel value is notfound in the detection area of the image 1. Thus, by performing stepsS331 to S335 again, the positional information of the pixel pointcorresponding to the maximum pixel value in the image is determinedagain. Specifically, it is preferred to traverse again (i.e., the secondtime) the area enclosed by the detection area in the detection imagetraversed in the step S331 (such as the area enclosed by the detectionarea in the image 1), thereby avoiding missing some pixel points in apart of the detection area so that the pixel point having the largestpixel value in the detection area could not be determined when the image1 is traversed during the first time. Thus, the image 1 is traversed asecond time, and the positional information of the pixel pointcorresponding to the maximum pixel value in the image is determined bythe steps S332 to S335 for the second time. The specific process is thesame as the description of steps S331 to S335, and the details thereofare not described again herein.

Further, in one embodiment, after the process of determining a pixelpoint correspond to a maximum pixel value in the image based on thetracking point, the method includes the following:

When the number of elements in the preset extreme value array is greaterthan one, all elements of the extreme value array are traversed. Then, adistance between each of pixel points corresponding to the elements andthe tracking point is respectively determined. Then, the distancebetween each of pixel points corresponding to the elements and thetracking point is compared to one another and a pixel point having thesmallest distance from the tracking point is acquired. Then, thecorresponding positional information of the pixel point having thesmallest distance is determined as the positional information of thepixel point corresponding to the maximum pixel value in the image.

In the embodiments of the present disclosure, for the extreme valuearray, if the number of elements in the extreme value array is 0, itmeans that the current frame of image has no extreme pixel value. Atthis time, image 1 is re-traversed to find an extreme pixel value, whichis then added to the extreme value array. When the number of elements inthe extreme value array is 1, it means that the value is the maximumpixel value. When the number of elements in the extreme value array isgreater than 1, it means that there is a plurality of maximum pixelvalues. The distance between each element and the tracking point iscalculated and compared, and the pixel point having the smallestdistance is obtained. The positional information of the pixel pointhaving the smallest distance is then determined as the positionalinformation of the pixel point corresponding to the maximum pixel value.

One example of the present disclosure is a gesture recognition method.The gesture recognition method includes determining whether a gestureshaking occurs using the gesture shaking recognition method according toany one of the embodiments of the present disclosure and, when thegesture shaking occurs, processing the next frame of image by imageparameters corresponding to the previous frame of image of the twoadjacent frames of images.

In one embodiment, after the gesture shaking is determined to haveoccurred based on the gesture image, the parameters corresponding to theimage of the next frame are updated by all parameters of the cameradevice capturing the current gesture image. That is, the image of thenext frame is also the image taken by the camera device. The imageparameters include, for example, a position of the selected trackingpoint in the embodiment of the present disclosure. For example, when itis determined that the gesture shaking does not occur, the position ofthe tracking point selected in the current frame of image is determinedas the position of the tracking point selected in the next frame ofimage. When the shaking occurs, the position of the tracking point inthe next frame of image is determined based on the difference disHandPosof the positions of the tracking points in the adjacent two frames ofimages. For example, when the difference disHandPos is −2, in the nextframe, the tracking point moves by two pixels to the right in a straightline. If the tracking point after the moving is not on the gesturecontour, the tracking point moves up or down to make the tracking pointon the gesture contour.

In the embodiments provided by the present disclosure, in combinationwith the foregoing description, the gesture recognition method includesthe following process. As shown in FIG. 7, first, the detection systemis initialized so that, for example, a frame rate of the camera, and avideo image size, etc. are set. The frame rate of the camera is set tofacilitate processing the images in sequence and to find the pixel pointhaving the maximum pixel value in the detection area of the image (i.e.,the extreme points in FIG. 7). The video image size is set to facilitateobtaining and processing an image of a suitable size. After the cameracaptures one frame of image, the camera device sends the frame of theimage into the gesture shaking recognition device to perform the gestureshaking recognition method according to one embodiment of the presentdisclosure. For example, Gaussian filtering is performed on the frame ofthe image to reduce image noise and avoid the influence of noise onextracting or segmenting the gesture image. Then, the frame of the imageis converted into the YCrCb color space, the Cr channel is extracted forprocessing, and the otsu algorithm is used to segment the area of thehand. That is, the pixel points on the gesture contour in the image areprocessed to have a same pixel value, and the remaining pixel pointsoutside the gesture contour are processed to have another pixel value,which has a large difference from the pixel value of the pixel points onthe gesture contour. The large difference facilitates the otsu algorithmto segment the gesture contour image. After the gesture contour image issegmented, the position of the tracking point in the current frame ofimage is obtained and the extreme value point of the gesture image isfound by using the preceding gesture shaking recognition methodaccording to one embodiment of the present disclosure. After the gestureimage of the current frame is determined, the image of the next frame iscaptured by the camera. Then, the foregoing process is performed on theimage of the next frame to acquire the tracking point and the extremevalue point of the gesture image of the next frame. Then, the differencebetween the positions of the tracking points in the two frames of imagesand the difference between the extreme value points in the two frames ofimages are obtained. In combination with the foregoing condition 1 andcondition 2, whether the gesture shaking in a frame of image occurs isdetermined. When the shaking occurs, based on the position of thetracking point of the current frame of the image and the difference ofpositions of the tracking points in the two frames of images, theposition of the tracking point in the next frame of the image isadjusted to eliminate shaking, thereby improving the stability of themotion trajectory of the gesture in the process of human-computerinteraction and the customers person-machine interaction experience.

One embodiment of the present disclosure provides a gesture shakingrecognition apparatus. As shown in FIG. 8, the gesture shakingrecognition apparatus includes an image acquisition module 100, atracking point selection module 200, a maximum pixel value acquisitionmodule 300, and a determination module 400.

The image acquisition module 100 is configured to acquire two adjacentframes of gesture images.

The tracking point selection module 200 is configured to select atracking point in each frame of gesture images, wherein the selectedtracking point in each frame of the gesture images corresponds to thesame position on the gesture.

The maximum value pixel point acquisition module 300 is configured todetermine, based on the tracking point, the positional information ofthe pixel point corresponding to a maximum pixel value in the image,that is, the position information of the pixel point having the maximumpixel value.

The determination module 400 is configured to determine whether thegesture shaking occurs based on the positional information of the pixelpoints corresponding to the maximum pixel values and the positionalinformation of the corresponding tracking points in the two adjacentimages.

Another embodiment of the present disclosure further provides a computerreadable storage medium storing a computer program that, when executed,implements the steps of the gesture shaking recognition method of anyone of the embodiments of the present disclosure.

Another embodiment of the present disclosure provides a gesturerecognition apparatus. In one embodiment, the apparatus includes aprocessor and a memory. The memory is used for storing a computerprogram, and when the computer program is executed by the processor, thesteps of the gesture shaking recognition method of any one of theembodiments of the present disclosure are implemented.

In another embodiment, the gesture recognition apparatus includes acamera device and a computer readable storage medium or a gestureshaking recognition device connected to the camera device. The cameradevice is configured to collect an image stream comprising a pluralityof the gesture images. The image stream is composed of a plurality offrames of images, wherein at least two frames of the images include agesture. The gesture image is the image formed after a frame of theimage in the image stream is processed.

In addition, each functional module in the embodiments of the presentdisclosure may be integrated into one processing module, or each modulemay exist physically separately, or two or more modules may beintegrated into one module. The above integrated modules can beimplemented in the form of hardware or in the form of softwarefunctional modules. The integrated modules, if implemented in the formof software functional modules and sold or used as separate products,may also be stored in a computer readable storage medium.

The storage medium mentioned above may be a read only memory, a magneticdisk or an optical disk or the like.

The principle and the embodiment of the present disclosures are setforth in the specification. The description of the embodiments of thepresent disclosure is only used to help understand the method of thepresent disclosure and the core idea thereof. Meanwhile, for a person ofordinary skill in the art, the disclosure relates to the scope of thedisclosure, and the technical scheme is not limited to the specificcombination of the technical features, and also should covered othertechnical schemes which are formed by combining the technical featuresor the equivalent features of the technical features without departingfrom the inventive concept. For example, technical scheme may beobtained by replacing the features described above as disclosed in thisdisclosure (but not limited to) with similar features.

1. A gesture shaking recognition method, comprising acquiring twoadjacent frames of gesture images of a gesture; selecting a trackingpoint in each of the two adjacent frames of gesture images, the trackingpoint in each of the two adjacent frames of gesture images correspondingto a same position on the gesture; determining positional information ofa maximum value pixel point in each of the two adjacent frames ofgesture images based on the tracking point; and determining whethergesture shaking occurs based on the positional information of themaximum value pixel point in each of the two adjacent frames of gestureimages and positional information of the tracking point in each of thetwo adjacent frames of gesture images.
 2. The gesture shakingrecognition method according to claim 1, wherein determining whethergesture shaking occurs based on the positional information of themaximum value pixel point in each of the two adjacent frames of gestureimages and the positional information of the tracking point in each ofthe two adjacent frames of gesture images comprises: determining a firstdifference between the positional information of the tracking point ineach of the two adjacent frames of gesture images; determining a seconddifference between the positional information of the maximum value pixelpoint in each of the two adjacent frames of gesture images; anddetermining whether the gesture shaking occurs based on the firstdifference and the second difference.
 3. The gesture shaking recognitionmethod according to claim 2, wherein determining whether the gestureshaking occurs based on the first difference and the second differencecomprises: determining that the gesture shaking occurs when a product ofthe first difference and the second difference is less than zero.
 4. Thegesture shaking recognition method according to claim 2, whereindetermining whether the gesture shaking occurs based on the firstdifference and the second difference comprises: determining that thegesture shaking occurs when a product of the first difference and thesecond difference is greater than or equal to zero, an absolute value ofthe first difference is less than a first threshold, and an absolutevalue of the second difference is less than a second threshold.
 5. Thegesture shaking recognition method according to claim 4, wherein thefirst threshold is 7 and the second threshold is
 5. 6. The gestureshaking recognition method according to claim 1, wherein determining thepositional information of a maximum value pixel point in each of the twoadjacent frames of gesture images based on the tracking point comprises:sampling each of the two adjacent frames of gesture images to obtain atleast one detection image for each of the two adjacent frames of gestureimages, the two adjacent frames of gesture images being as originaldetection images; constructing a detection area centering on thetracking point in each of the at least one detection image and each ofthe original detection images; and comparing all pixel values of pixelpoints in the detection area sequentially to determine the maximum valuepixel point in each of the two adjacent frames of gesture images.
 7. Thegesture shaking recognition method according to claim 6, whereincomparing all pixel values of pixel points in the detection areasequentially to determine the maximum value pixel point in each of thetwo adjacent frames of gesture images comprises: traversing a currentpixel point in a detection area in one of the at least one detectionimage and the original detection images; acquiring a plurality of pixelpoints adjacent to the current pixel point traversed, the current pixelpoint being a center of the plurality of pixel points; acquiring anotherpixel point and another plurality of pixel points adjacent to theanother pixel point in each of the remaining detection image andoriginal detection images, the another pixel point corresponding to thecurrent pixel point, and the another plurality of pixel pointscorresponding to the plurality of pixel points adjacent to the currentpixel point respectively; and determining whether a pixel value of thecurrent pixel point is the largest among pixel values of the pluralityof pixel points adjacent to the current pixel point, a pixel value ofthe another pixel point and pixel values of the another plurality ofpixel points in each of the remaining detection image and originaldetection images.
 8. The gesture shaking recognition method according toclaim 7, further comprising, when the pixel value of the current pixelpoint is the largest, extracting positional information of the currentpixel point into an extreme value array, the current pixel point beingan extreme value pixel point, and the position information of thecurrent pixel point being an element of the extreme value array.
 9. Thegesture shaking recognition method according to claim 8, furthercomprising: sequentially comparing pixel values corresponding toelements in the extreme value array to determine the positionalinformation of the maximum value pixel point in each of the two adjacentframes of gesture images.
 10. The gesture shaking recognition methodaccording to claim 8, wherein when the extreme value array has zeroelement, the position information of extreme value pixel point isre-determined based on the tracking point.
 11. The gesture shakingrecognition method according to claim 8, wherein sequentially comparingpixel values corresponding to elements in the extreme value array todetermine the positional information of the maximum value pixel point ineach of the two adjacent frames of gesture images comprises: traversingall elements of the extreme value army, the number of elements in theextreme value array being greater than one; determining a distancebetween each pixel point corresponding to each of the elements and thetracking point; and comparing the distance between each pixel pointcorresponding to each of the elements and the tracking point to acquirea pixel point having the smallest distance, and the pixel point havingthe smallest distance being the maximum value pixel point.
 12. Thegesture shaking recognition method according to claim 6, beforeacquiring two adjacent frames of gesture images of the gesture,comprising: acquiring two adjacent frames of original images of thegesture, and constructing a square area as the detection area with acenter being the tracking point and a side length being one-fifth of aside length of an original image box.
 13. A gesture recognition method,comprising: the gesture shaking recognition method according to claim 1to determine whether a gesture shaking occurs.
 14. The gesturerecognition method of claim 13, further comprising: when the gestureshaking occurs, for the two adjacent frames of images comprising aprevious frame of image and a next frame of image, the next frame ofimage is processed by image parameters corresponding to the previousframe of image.
 15. A gesture shaking recognition apparatus comprising:an image acquisition module configured to acquire two adjacent frames ofgesture images of a gesture; a tracking point selection moduleconfigured to select a tracking point in each of the two adjacent framesof gesture images, wherein the tracking point in each of the twoadjacent frames of gesture images corresponding to a same position onthe gesture; a maximum value pixel point acquisition module configuredto determine, based on the tracking point, positional information of amaximum value pixel point in each of the two adjacent frames of gestureimages; and a determination module configured to determine whether thegesture shaking occurs based on the positional information of themaximum value pixel point and the position information of the trackingpoint in each of the two adjacent frames of the gesture images.
 16. Acomputer readable storage medium storing a computer program that, whenexecuted, implements the steps of the gesture shaking recognition methodaccording to claim
 1. 17. A gesture shaking recognition apparatus,comprising a processor and a memory, the memory being configured tostore a computer program that, when the computer program is executed bythe processor, implements the steps of the gesture shaking recognitionmethod according to claim
 1. 18. A gesture recognition apparatus,comprising a camera device and the gesture shaking recognition apparatusof claim 15 being connected to the camera device, wherein the cameradevice is configured to collect an image stream comprising the twoadjacent frames of the gesture images.